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j i 1 . Introduction 

;'j  Consider  the  d-dimensional  mixed  autoregressive  moving  average 

f • 

process  {x(t)  , t € z}  , Z the  set  of  integers,  of  order  (p,q)  , 

S 

p q 

E A(j)  X(t-j)  = E B(k)  e(t  - k)  , t€Z 
1 j=0  ~ k=0 

where  A(0)  = B(0)  e , the  d-dimensional  identity  matrix,  and 

{e(t)  , t € z}  is  a collection  of  uncorrelated,  zero  mean,  d-dimensional 

random  variables  each  having  positive  definite  covariance  matrix  ^ . 

The  estimation  of  the  (d  x d)  matrices  A(l) , . . . , A(p)  , 

B(l) , . . . ,B(q)  , and  ^ has  received  considerable  attention  in  the  recent 
literature  (Akaike  (1973a),  Wilson  (1973),  Dunsmuir  and  Hannan  (1976),  for 
example).  One  of  the  most  difficult  computational  problems  involved  in 
the  estimation  is  that  of  determining  the  asymptotic  covariance  matrix 
V of  the  maximum  likelihood  estimators.  Hannan  (1970),  p.  385  and  p.  329 
has  given  expressions  for  V for  the  case  p = 0 or  q = 0 . 

In  this  paper  we  derive  closed  form  matrix  expressions  for  V by 
using  Whittle's  (1953)  formula  for  the  Fisher  information  matrix  of  the 
parameters  of  a Gaussian  multiple  time  series. 


2. 


The  Information  Matrix  of  the  Mixed  Process 


Let  {Y(t)  , tez)  be  a zero  mean  Gaussian  time  series  whose  dis- 

T T 

tribution  depends  on  parameters  0 = , where  A denotes 

the  transpose  of  the  matrix  A . Let  f(u))  , U)e[-TT,TT]  be  the  spectral 
density  matrix  of  Y(*)  , Then  given  a sample  realization  y(l) , . , . ,y(T) 
of  Y(*)  , Whittle  (1953)  shows  that  the  maximum  likelihood  estimators 
6^  of  0 are  such  that  (6^  * 6)  is  asymptotically  r-dimensional 

normal  with  mean  zero  and  covariance  matrix  V(0)  = I ^(0)  where  the 
(j,k)th  element  of  1(0)  is  given  by 


^ I “ffff 

-TT  L j k J 


du)  , 


(2.1) 


9A 

where  the  notation  — , b a scalar  and  A = (A.,  ) and  (n  x m)  matrix, 

ob  jk 


denotes  the  (n  x m)  matrix  of  derivatives 
trace  of  the  matrix  B . 


and  tr  B denotes  the 


Let  D = (D  ) be  an  (n  x n)  matrix  and  U = U(D)  , V = V(D)  be 

jk 

matrix  functions  of  D , Then  (Neudecker  (1969),  for  example) 


= F IJDL-  = F^  - E 

“'jk  > ‘'jk 


(2.2) 


where  E . is  the  zero  matrix  except  the  (j,k)th  element  which  is  one. 
jk 

Also,  the  chain  rule  holds; 


3 


Thus  from  D D = I , where  I denotes  the  nth  order  identity  matrix, 
n n 

we  have 


(2.4) 


Note  that  the  trace  operation  satisfies 


tr  (A  + B)  = tr  (A)  + tr  (B) 
tr  (AB)  = tr  (BA) 


From  (2.4)  we  can  write  (2.1)  as 


(2.5) 


(2.6) 


where  we  have  now  deleted  the  argument  of  f(*)  for  convenience. 

The  spectral  density  matrix  f(*)  of  the  mixed  model  can  be  written 
(see  Hannan  (1970),  p.  67) 


f(tu) 


2tt 


G-^e^'") 


where  the  complex  matrix 


H(e^“^)  t G-*(e"“^)  , 

polynomials  G(*)  and  H(*) 


(i)e[-n,TT] 


are  given  by 


(2.7) 
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G(z)  = E A(j) 
J=0 


H(z)  = E B(k)  z 
k=0 


and  A denotes  the  complex  conjugate  transpose  of  the  matrix  A 


We 


assume  that  the  zeros  of  det 


^G(z)^  and  det  ^H(z)^ 


are  outside  the 


unit  circle  so  that  the  elements  of  G ^(z)  and  H ^(z)  can  be  written 


as  power  series  in  z . 


From  (2.7)  we  write 


-1  . * -*  -1  -* 
2TTf  = G H $ H G s G QG 


(2.8) 


^ f"^  = G*q’’-G  , (2.9) 

where  G doesn't  involve  the  B(*)  , Q = h;|]H  is  Hermitian  and  mathe- 
matically independent  of  the  A(*),  Q=$  if  9=0,  and  we  have 
deleted  the  arguments  of  all  functions. 

Clearly  we  must  order  the  elements  of  the  A(«)  and  B(*)  into  a 
vector  9 . However,  we  first  find  the  element  of  the  information  matrix 
corresponding  to  Ajj^(v)  , A^(u)  (denoted  I[Ajj^(v)  , ) l>y  (2.5), 

the  element  corresponding  to  B^^(v)  , B^(u)  (denoted  I[B^j^(v)  , ) 

by  (2.1),  and  the  element  corresponding  to  Ajj^(v) , B^^(u)  (denoted 
1[ Ajj^(v) , B^^(u)  by  (2,6)  ).  Then  we  consider  various  orderings  of  the 
elements  of  the  A(*)  and  B(*)  to  find  a convenient  expression  for  1(0)  . 


i 


From  (2.2),  (2,3) 


and  (2.5)  we  obtain 


1 Sf 


-1 


2tt  ^Ajj^(v) 


a * -1 


0 + G*Q-1 

aA.k(v)  SAjk(v) 


= EkjQ-'Ge-^-HcV'Ej^e^"^ 


Thus 


&Ajk(v)  kj  jk 


and 


hf'}  f _a£i 


» /N'-iA  ,G  E.G 

5Ajk(v)  dA^(u)  kj  m-L 


-*  -i(u-W)a,^2TTE,  O-^E, 

kj  -un 


* -1  -*  -i(u-v)U) 

+ 2nGQ  'E^k^E^^G  e 


jk  -(m 


Denote  by  the  matrix  D replaced  by  zeros  except  for  the 

r^*’  row  which  is  replaced  by  the  s^^  row,  i.e.  d''®  = E^^D  . Then 
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for  matrices  D 


- '“jk’ 


(C^)  we  have 


trfE  DE^  C]  = tr[D^®C*^“]  = D 

rs  tu  St  ur 


Thus 


tr 


a 


hf 


-1 


hf' 


f 


-* 

G E ,G 

m’O 


g-i(u+v)u) 


+ 2^  e-l(“-’')«> 


+ trIE,  G-‘e.,G-'i 

-un  jk 


9^  n"^  f _-i(v-u)uo 


+ 2tt  Q 


-1 

^3 


-i(u'v)u)  , „-l  p-1  ^i(u+v)a' 
r,  e + G . G,  . e 

km  mj  k-L 


We  argue  that  the  first  and  last  terms  integrate  to  zero,  as  follows: 

because  the  roots  of  det  ^G(z)^  = 0 are  assumed  strictly  outside  the 

unit  circle,  G ^ , the  (j,k)th  element  of  G ^ , can  be  written  as  a 
J k 

power  series  (which  allows  the  interchange  of  summation  and  integration 
in  (2.10)  below) 


7 


and  thus 


-TT 


-1  „-l  i(u+v)tju. 


TJ  CO  00 


-J  7:.  S^<'> 


i(u+v+t+r)uu 


-TT  t=0  r=0 


du) 


00  00 


t=0  r=0  -TT 


^ i(u+v+t+r)tju 


duu  , 


and  the  integral  is  always  zero  since  (u  + v + t+r)  > 0 


Thus  we  are  left  with 


I tr[C  + C*]  duj  , 


4tt 


-n 


where 


C = 2ttE  Q"^E  fe-i(v-u)uj  . 
kj  Im 


Since  the  integral  is  real,  we  have 


= ^ 1 2tr[C]  du) 


-TT 


- 1 o;iu 

-TT 


(2.10) 


(2.11) 


From  (2.8)  we  have 


2tt 


Sf 


SBjk(v) 


. g'^-H  X hV* 

SB.k(v) 


5g'^h  ^ * -*  -1  5 X h*g'* 


-1  -t.  * IVID  -I  a.  1 

= G t:  H G e + G H t G e 


Then 


. . f"^  = g‘^E  X hV’’  E.  X Qgi(u+v)u) 

aB„  (v)  dB,  (u)  jk  ^ ^ -Cm  ^ 

jk  'em 


+ G-'Ej^fHV‘HtE,^q-*G.-‘<"-''>“’ 


+ G'l  H t E^jQ"'  E^  t H*q-lGe'‘<’''“>"' 


+ G-lHt  E^jq-lHtE^^q-lG.-^'"-^)”' 


The  trace  of  this  is  given  by 
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where 


C 


E,  h‘^  E.,  H 
-un  jk 


-1 


i(u+v)uj 

6 


D 


^jk 


t 


^-i(u-v)uj 


The  integral  of  the  first  term  again  vanishes,  and  we  have 


I[B  (v).B  (u)]  = ^ 

jk  -un  2tt 


TT 

J 

-TT 


(2.12) 


To  find  the  off  block  diagonal  elements  of  I we  obtain,  using 
arguments  similar  to  the  ones  above,  that 


tr 


Sf~^  hf 


= tr[C  + 


C ] 


+ tr[D  + 


D*] 


where 


C 


E h‘^E  g"^ 
‘'jk^  ® 


D 


t K G-* 


Thus 


~TT 


[ t:  H*  g' 


-i(v-u)u) 


mk 


du) 


(2.13) 
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where  the  (m,k)th  element  of  the  product  ^ H G is  denoted 


, * -* 

[ t H G ] 


mk 


So  the  integrands  in  the  expressions  for  1 (Ajj^(v) , A^(u)  ] , 

(u)]  , and  I[A  (v),Bj,  (u)]  are  all  of  the  form 
Jk  ^ jk  -c-m 

^ jl  ^mk  ^ some  matrices  C and  D , i.e. . we  have  the  trans- 

formation of  indices  denoted 


(j.k.v)  , (-t.m.u)  (j.-t.-v)  , (m,k,u)  . 


(2.14) 


This  transformation  allows  us  to  find  general  expressions  for  the  informa- 
tion matrix  for  various  orderings  of  the  elements  of  the  A(*)  and  B(*) 
matrices. 


Ordering  of  A(*),  BC*’ 


Define  the  partitioned  matrices 


= ^A(l)  • . . . • A(p)j 

A^  = ^A^(l)  : • • • : A^(p)j  , 


B^  = B(l) 


= ^B^(l)  • • • • : B^(q)^ 
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Then  let 


ttj  = vec  (Aj) 


a2  = vec  (Aj^) 


^ = vec  (A^) 


= vec  (B^) 


P = vec  (Bn 


Pj  = vec  (Bj^) 


" ei(j-k)lU  ^ j = 1. . , . .n,k  = 1, . . . .m  , 


where  vec  (A)  denotes  the  operation  of  stacking  the  columns  of  A , i.e., 

if  ^ Gl  i ’ * * i ~1)  ’ 

Define  the  Kronecker  product  of  the  nxm  matrix  A = (A,,  ) and 

the  rxs  matrix  B = (B.,)  as  the  nrxms  matrix 

J*^ 


C = A ® B 


A,,B  • • • A,  B 
11  Im 


A ,B  • • • A B 
nl  nm 


Denote  the  information  matrix  of  parameters  0. ,0-  by  I . Note 

®2®1  f T T T 

that  1 = \1  ^ , and  that  (C  ® D)  = C ® D . Then  we  have 


from  (2.11),  (2.12),  and  (2.13), 
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We  have  derived  the  information  matrices  for  three  methods  of  order- 
ing the  elements  of  the  parameter  matrices.  The  results  for  and 

are  used  below  to  find  a convenient  form  for  the  information  matrix  of 
the  parameters  of  a mixed  scheme.  The  results  for  and  a2  are  used 

for  autoregressive  processes,  while  those  for  are  used  for  moving 

average  processes.  To  summarize,  we  have 

Remarks  on  Information  Matrices 


-TT 


where 


X = (2TT)'^  Jp  ® ® h"* 

Y = -(2tt)’^  J ® 0 h'*  , 

'•H 


jk 

Z denotes  a positive  definite  square  root  of  the  positive  definite 

matrix  Z , and  is  an  r-dimensional  vector  whose  kth  element  is 

-ikuj 

e 


b)  To  find  the  information  matrix  of  the  elements  of  ^ , and  the 
cross  information,  in  turn,  between  the  A(*)  and  ^ , and  the  B(*) 
and  ^ , we  note  that  since  ^ is  symmetric 


3^  ■ 


where  C = G , and  = 1 if  j = k and  0 otherwise.  Defining 


jk 


(1 


we  have 


tr 


tr  [D .. 
jk 


which  is  independent  of  UJ  . Thus  the  element  of  the  information  matrix 
corresponding  to  » denoted  ^-jk’  , is  given  by 
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1^(2 -6 


jk  ^-Cm  ^ ^jk^-C-m^ 


^nk 


+ 


(2.15) 


where  we  consider  t , u s v = l,...,d  as  the  distinct  elements  of 
T"u,v 

^ . To  write  this  in  matrix  form,  define 


Oj^  = vec  ( ^ ) 

£2  = 't'Vec  ( $ ) , 


where  -tvec  (A)  is  the  vec  operation  on  the  lower  triangular  portion  of 
the  matrix  A . Thus  O2  contains  the  distinct  elements  of  ^ . 


2 

If  we  treat  the  d elements  of  ^ as  being  distinct  (i.e., 
ignore  the  symmetry  of  ^),  then  D.,  = E , and  it  is  easy  to  show 

JX  JK 

that 


I 


QlGi 


Note  that  we  can  write 


^1  ~ ^^2 


15 


where  A is  a x matrix  of  zeros  and  ones.  Then 

(Pagano  (1974)) 


^2^2  T *^1*^1 
= AI  A 


A^(  ® t'^)  A 


(2.16) 


Thus  the  information  matrix  of  can  be  found  from  either  (2.15)  or 

(2.16). 


c)  One  can  also  show  that 


IfAjk(v),  t^] 


= 0 , 


and 


dcu 


0 , 


These  Integrals  are  zero  for  the  same  reason  that  the  integrals  vanish 
in  the  derivation  of  the  information  matrices  of  the  A(»)  and  the 


B(*)  . 
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d)  If  q = 0 we  have 

(2.17) 

(2.18) 

since 


^ ^ = r ® t 

p 

® r 

^ p 


Pp  s BTOEPL  (R(0),R(-l),...,R(l-p)) 


TT 

J ® f du)  , 
PP 


where  BTOEPL  (R(0) , . . . ,R(1  - p))  is  a block  Toeplitz  matrix  having 
R(j  - k)  in  the  jth  row  and  kth  column  of  blocks,  j , k = l,...,p  . 
Note  that  (2.18)  is  the  result  given  by  Hannan  (1970),  p.  329. 


-T  1 -T 

e)  If  p = 0 , then  Q ~ ^ ^ which  is  of  the  form  of  an 


autoregressive  spectral  density  matrix.  Thus  I 


Pl^l 


should  be  of  the 


ttitti 

same  form  as  I of  the  previous  note  with  the  autocovariances 

-T 

corresponding  to  the  spectral  density  Q replacing  R(*)  and 
,-l 


2rr  replacing  . Thus 


(2.19) 


where 


riq  * BTOEPL^Ri(O),. . . ,Ri(l  - q)^  , 

TT 

Ri(v)  = — ^ r f ^(uu)  , v6Z  . 

-tt 

See  Newton  (1975),  p.  11,  et  seg  for  a discussion  of  the  inverse  auto- 
covariances  Ri(*)  . Note  that  (2.19)  is  the  result  stated  by  Hannan 
(1970),  p.  385. 

f)  Equations  (2.18)  and  (2.19)  also  provide  an  interpretation  of 
information  matrices  of  random  matrices  (transformed  into  a random 
vector  by  the  vec  operator)  as  the  covariance  matrix  of  some  random 
vector.  We  can  write 

I = {R(j  -k)  ® t 

•'  ^ pxp 


and 


PiP 


1^1 


-1 


= {Ri(j  -k)  ® $.  } 


qxq 


^1^1  2 2 ^1^ 
i.e. . I is  a (p X p)  block  matrix  of  (d  x d ) blocks,  while  1 

2 2 

Is  a (qxq)  block  matrix  of  (d  x d ) blocks.  Thus  there  exist 
2 

d -dimensional  random  vectors  z,,...,z  and  y, , . . . ,y  such  that 

/ T • • T\  ^1°'1 

z = ^Zj^  • •••  j z^j  had  covariance  matrix  given  by  I and 

/ T • • T\  ^1^1 

y = ^yj^  • •••  I kas  covariance  matrix  given  by  I 


g)  Akaike  (1973a)  derives  approximate  expressions  for  the  elements 
of  the  Hessian  of  the  log  likelihood  function.  From  the  formulas  (2.11), 
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(2.12),  (2.13)  above,  it  is  clear  that  Akaike's  formulas  for  the  Hessian 
are  the  sample  analogues  of  the  information  matrix  of  the  mixed  scheme 
parameters.  Thus  one  can  use  the  block  Toeplitz  matrix  inversion  tech- 
niques developed  by  Akaike  (1973b)  to  find  the  asymptotic  covariance 
matrix  of  the  maximum  likelihood  estimators. 
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